Ensemble Feature Selection Approach Research Articles

A robust ensemble feature selection approach to prioritize genes associated with survival outcome in high-dimensional gene expression data

Exploring features associated with the clinical outcome of interest is a rapidly advancing area of research. However, with contemporary sequencing technologies capable of identifying over thousands of genes per sample, there is a challenge in constructing efficient prediction models that balance accuracy and resource utilization. To address this challenge, researchers have developed feature selection methods to enhance performance, reduce overfitting, and ensure resource efficiency. However, applying feature selection models to survival analysis, particularly in clinical datasets characterized by substantial censoring and limited sample sizes, introduces unique challenges. We propose a robust ensemble feature selection approach integrated with group Lasso to identify compelling features and evaluate its performance in predicting survival outcomes. Our approach consistently outperforms established models across various criteria through extensive simulations, demonstrating low false discovery rates, high sensitivity, and high stability. Furthermore, we applied the approach to a colorectal cancer dataset from The Cancer Genome Atlas, showcasing its effectiveness by generating a composite score based on the selected genes to correctly distinguish different subtypes of the patients. In summary, our proposed approach excels in selecting impactful features from high-dimensional data, yielding better outcomes compared to contemporary state-of-the-art models.

Heterogeneous Ensemble Feature Selection and Multilevel Ensemble Approach to Machine Learning Phishing Attack Detection

Over the decade, technology has presented human facets with easiest means of accomplishing complex tasks seamlessly, especially in the area of communication. Malicious and vicious links are consciously doctored to resemble the original and sent through emails to millions of users at once at a lower price. Since the emergence of phishing and its cohorts, every solution and means to mitigate the attacks has proven unsuccessful due to the dynamic nature of the attacks. Meanwhile, machine learning (ML) is adopted as the right antidote to phishing detection, with its performance based on diverse steps, especially feature selection. Most studies in the problem domain concentrate more on model optimization than sourcing for a reliable feature selection system and fail to integrate a reliable feature selection along with the classification model. The systems are fed with low-quality data that hampers the performance of such models. The authors noticed the contribution of feature selection to the performance of machine learning models and developed a novel Heterogeneous Ensemble Feature Selection (HEFS) framework for multilevel ensemble machine learning-based phishing detection. In HEFS, three filter-based statistical techniques were exploited to produce a primary subset of phishing features, and the variable selected by each of the techniques was automatically aggregated to produce the baseline features. The selection of the techniques is to overcome each limitation since their ranking principles are different. The experiment revealed that the multilevel ensemble (stacked) on the baseline features outperformed others with an accuracy of 98.8%., including multilevel model on each filter-based method.

Detecting Network Intrusion in Cloud Environment Through Ensemble Learning and Feature Selection Approach

Detecting Network Intrusion in Cloud Environment Through Ensemble Learning and Feature Selection Approach

Machine Learning Analysis of Alzheimer's Disease Single-Cell RNA-Sequencing Data across Cortex and Hippocampus Regions.

Advancements in molecular biology have revolutionized our understanding of complex diseases, with Alzheimer's disease being a prime example. Single-cell sequencing, currently the most suitable technology, facilitates profoundly detailed disease analysis at the cellular level. Prior research has established that the pathology of Alzheimer's disease varies across different brain regions and cell types. In parallel, only machine learning has the capacity to address the myriad challenges presented by such studies, where the integration of large-scale data and numerous experiments is required to extract meaningful knowledge. Our methodology utilizes single-cell RNA sequencing data from healthy and Alzheimer's disease (AD) samples, focused on the cortex and hippocampus regions in mice. We designed three distinct case studies and implemented an ensemble feature selection approach through machine learning, also performing an analysis of distinct age-related datasets to unravel age-specific effects, showing differential gene expression patterns within each condition. Important evidence was reported, such as enrichment in central nervous system development and regulation of oligodendrocyte differentiation between the hippocampus and cortex of 6-month-old AD mice as well as regulation of epinephrine secretion and dendritic spine morphogenesis in 15-month-old AD mice. Our outcomes from all three of our case studies illustrate the capacity of machine learning strategies when applied to single-cell data, revealing critical insights into Alzheimer's disease.

Bio-Inspired ensemble feature selection and deep auto-encoder approach for rapid diagnosis of breast cancer

Bio-Inspired ensemble feature selection and deep auto-encoder approach for rapid diagnosis of breast cancer

Impact of the COVID-19 pandemic on the stock market and investor online word of mouth

Investor sentiment based on social media Word-of-Mouth (WoM) has been shown to be predictive of stock returns under normal market conditions. However, the COVID-19 pandemic has brought unprecedented impact and uncertainty to the stock market. In this paper, we examine the predictability of social media-based WoM on day-ahead US stock market returns across three phases of the pandemic: pre-COVID, peak-COVID, and recovery. We collect over 24 million stock-related tweets from Twitter across these three periods and employ text-mining methods for feature engineering. We focus on two channels of WoM information, aggregate sentiments, and disaggregated topics, representing investors' overall opinions and discussion patterns about the stock market. We find that, in contrast to normal market conditions, the investor sentiment indices are not significant in predicting market returns during the peak-COVID and recovery periods, while for the pre-COVID period, our findings are consistent with prior studies. To explore the effect of disaggregated latent topics, we propose an ensemble feature selection approach that identifies relevant topics for each of the three periods. We find that adding topic information significantly improves model prediction during the peak-COVID period. Our findings imply that during a global socio-economic disruption and the subsequent recovery, the landscape of financial markets, examined through the lens of social media-based WoM, is different than during normal market conditions. Our findings also suggest that investors who primarily trade on sentiment may wish to leverage machine learning techniques to identify latent topic information, subsequently incorporating such information into their trading decision-making.

Identifying predictors of analyst rating quality: An ensemble feature selection approach

Forecasting the analyst rating quality (ARQ), defined as whether a stock rating provided by an analyst can correctly foretell the stock movement, is crucial to fully leveraging the value of this information resource. This study develops a two-phase method to identify key predictors for ARQ forecasting. In the first stage, we conduct a thorough literature review to obtain a comprehensive list of candidate features, and organise them under three categories: analyst-related, rating-related, and stock-related. In the second stage, we propose a heterogeneous community-based ensemble feature selection method (ComEFS), with the goal of identifying a subset of relevant predictors to be jointly used for ARQ forecasting. Thorough experiments are conducted on a real dataset to verify the effectiveness of our proposed method. The empirical results show that key predictors identified by ComEFS exhibit stronger predictive power compared to those identified by benchmark methods. This study provides insights about ARQ forecasting by selecting the right input. Selectively utilizing these predictive features can help improve the performance of downstream machine learning models and ultimately help investors avoid unreliable analyst ratings and financial loss.

Solar Radiation Forecasting Using Machine Learning and Ensemble Feature Selection

Accurate solar radiation forecasting is essential to operate power systems safely under high shares of photovoltaic generation. This paper compares the performance of several machine learning algorithms for solar radiation forecasting using endogenous and exogenous inputs and proposes an ensemble feature selection method to choose not only the most related input parameters but also their past observations values. The machine learning algorithms used are: Support Vector Regression (SVR), Extreme Gradient Boosting (XGBT), Categorical Boosting (CatBoost) and Voting-Average (VOA), which integrates SVR, XGBT and CatBoost. The proposed ensemble feature selection is based on Pearson coefficient, random forest, mutual information and relief. Prediction accuracy is evaluated based on several metrics using a real database from Salvador, Brazil. Different prediction time-horizons are considered: 1 h, 2 h and 3 h ahead. Numerical results demonstrate that the proposed ensemble feature selection approach improves forecasting accuracy and that VOA performs better than the other algorithms in all prediction time horizons.

Exposure to diesel engine exhaust and alterations to the Cys34/Lys525 adductome of human serum albumin

We investigated whether exposure to carcinogenic diesel engine exhaust (DEE) was associated with altered adduct levels in human serum albumin (HSA) residues. Nano-liquid chromatography-high resolution mass spectrometry (nLC-HRMS) was used to measure adducts of Cys34 and Lys525 residues in plasma samples from 54 diesel engine factory workers and 55 unexposed controls. An untargeted adductomics and bioinformatics pipeline was used to find signatures of Cys34/Lys525 adductome modifications. To identify adducts that were altered between DEE-exposed and unexposed participants, we used an ensemble feature selection approach that ranks and combines findings from linear regression and penalized logistic regression, then aggregates the important findings with those determined by random forest. We detected 40 Cys34 and 9 Lys525 adducts. Among these findings, we found evidence that 6 Cys34 adducts were altered between DEE-exposed and unexposed participants (i.e., 841.75, 851.76, 856.10, 860.77, 870.43, and 913.45). These adducts were biologically related to antioxidant activity.

Novel Ensemble Feature Selection Approach and Application in Repertoire Sequencing Data.

The T and B cell repertoire make up the adaptive immune system and is mainly generated through somatic V(D)J gene recombination. Thus, the VJ gene usage may be a potential prognostic or predictive biomarker. However, analysis of the adaptive immune system is challenging due to the heterogeneity of the clonotypes that make up the repertoire. To address the heterogeneity of the T and B cell repertoire, we proposed a novel ensemble feature selection approach and customized statistical learning algorithm focusing on the VJ gene usage. We applied the proposed approach to T cell receptor sequences from recovered COVID-19 patients and healthy donors, as well as a group of lung cancer patients who received immunotherapy. Our approach identified distinct VJ genes used in the COVID-19 recovered patients comparing to the healthy donors and the VJ genes associated with the clinical response in the lung cancer patients. Simulation studies show that the ensemble feature selection approach outperformed other state-of-the-art feature selection methods based on both efficiency and accuracy. It consistently yielded higher stability and sensitivity with lower false discovery rates. When integrated with different classification methods, the ensemble feature selection approach had the best prediction accuracy. In conclusion, the proposed novel approach and the integration procedure is an effective feature selection technique to aid in correctly classifying different subtypes to better understand the signatures in the adaptive immune response associated with disease or the treatment in order to improve treatment strategies.

Ensemble Feature Selection Approach for Imbalanced Textual Data Using MapReduce

Feature selection is a fundamental pre-processing phase in text classification. It speeds up machine learning algorithms and improves classification accuracy. In big data context, feature selection...

PSO Based Optimized Ensemble Learning and Feature Selection Approach for Efficient Energy Forecast

Swarm intelligence techniques with incredible success rates are broadly used for various irregular and interdisciplinary topics. However, their impact on ensemble models is considerably unexplored. This study proposes an optimized-ensemble model integrated for smart home energy consumption management based on ensemble learning and particle swarm optimization (PSO). The proposed model exploits PSO in two distinct ways; first, PSO-based feature selection is performed to select the essential features from the raw dataset. Secondly, with larger datasets and comprehensive range problems, it can become a cumbersome task to tune hyper-parameters in a trial-and-error manner manually. Therefore, PSO was used as an optimization technique to fine-tune hyper-parameters of the selected ensemble model. A hybrid ensemble model is built by using combinations of five different baseline models. Hyper-parameters of each combination model were optimized using PSO followed by training on different random samples. We compared our proposed model with our previously proposed ANN-PSO model and a few other state-of-the-art models. The results show that optimized-ensemble learning models outperform individual models and the ANN-PSO model by minimizing RMSE to 6.05 from 9.63 and increasing the prediction accuracy by 95.6%. Moreover, our results show that random sampling can help improve prediction results compared to the ANN-PSO model from 92.3% to around 96%.

An Ensemble Feature Selection Approach to Identify Relevant Features from EEG Signals

Identifying relevant data to support the automatic analysis of electroencephalograms (EEG) has become a challenge. Although there are many proposals to support the diagnosis of neurological pathologies, the current challenge is to improve the reliability of the tools to classify or detect abnormalities. In this study, we used an ensemble feature selection approach to integrate the advantages of several feature selection algorithms to improve the identification of the characteristics with high power of differentiation in the classification of normal and abnormal EEG signals. Discrimination was evaluated using several classifiers, i.e., decision tree, logistic regression, random forest, and Support Vecctor Machine (SVM); furthermore, performance was assessed by accuracy, specificity, and sensitivity metrics. The evaluation results showed that Ensemble Feature Selection (EFS) is a helpful tool to select relevant features from the EEGs. Thus, the stability calculated for the EFS method proposed was almost perfect in most of the cases evaluated. Moreover, the assessed classifiers evidenced that the models improved in performance when trained with the EFS approach’s features. In addition, the classifier of epileptiform events built using the features selected by the EFS method achieved an accuracy, sensitivity, and specificity of 97.64%, 96.78%, and 97.95%, respectively; finally, the stability of the EFS method evidenced a reliable subset of relevant features. Moreover, the accuracy, sensitivity, and specificity of the EEG detector are equal to or greater than the values reported in the literature.

An ensemble feature selection approach using hybrid kernel based SVM for network intrusion detection system

Feature selection is a process of identifying relevant feature subset that leads to the machine learning algorithm in a well-defined manner. In this paper, anovel ensemble feature selection approach that comprises of Relief  Attribute Evaluation and hybrid kernel-based support vector machine (HK-SVM) approach is proposed as a feature selection method for network intrusion detection system (NIDS). A Hybrid approach along with the combination of Gaussian and Polynomial methods is used as a kernel for support vector machine (SVM). The key issue is to select a feature subset that yields good accuracy at a minimal computational cost. The proposed approach is implemented and compared with classical SVM and simple kernel. Kyoto2006+, a bench mark intrusion detection dataset,is used for experimental evaluation and then observations are drawn.

New hybrid data mining model for prediction of Salmonella presence in agricultural waters based on ensemble feature selection and machine learning algorithms

AbstractThis paper aims to create a new hybrid ensemble data mining model to predict the Salmonella presence in agricultural surface waters based on the combination of heterogeneous ensemble approach for feature selection, clustering, regression, and classification algorithms. The data set for this study was collected from six agricultural ponds in Central Florida consisting of 23 features with 540 instances (26 Salmonella positive and 514 Salmonella negative). The model consisted of three stages. Initially, a heterogeneous ensemble feature selection (HEFS) approach was applied to select top features. Then, the k‐means clustering algorithm was implemented to remove misclassified cases from the data set. Finally, classification and regression algorithms, including support vector machine (SVM), Naïve Bayes (NB), Artificial Neural Network (ANN), Random Forest (RF) with soft voting approach were applied to the preprocessed data set to predict the Salmonella presence in agricultural surface waters with the amount of test set (20%). These algorithms were combined in 10 different ensemble models through the soft voting approach. The performance of these hybrid ensemble models was also evaluated. The ensemble ANN + RF model achieved the highest performance and outperformed all other single and ensemble models based on Area under the ROC Curve (AUC) (0.98) and prediction accuracy (94.9%). The findings emphasize the validity of our hybrid ensemble model which encourages researchers to predict Salmonella presence in agricultural surface waters.

R-HEFS: Rough set based heterogeneous ensemble feature selection method for medical data classification

Feature selection is one of the trustworthy processes of dimensionality reduction technique to select a subset of relevant and non-redundant features from large datasets. Ensemble feature selection (EFS) approach is a recent technique aiming at accumulating diversity in the subset of selected features. It improves the performance of learning algorithms and obtains more stable and robust results. In this paper, a novel rough set theory (RST) based heterogeneous EFS method (R-HEFS) is proposed for selecting the less redundant and highly relevant features during the aggregation of diverse feature subsets by applying the feature-class, feature-feature rough dependency and feature-significance measures. In R-HEFS five state-of-the-art RST based filter methods are used as a base feature selectors. Experiments are carried out on 10 benchmark medical datasets collected from the UCI repository. For the imputation of the missing values and discretization of the continuous features, k nearest neighbor (kNN) imputation method and RST based discretization techniques are applied. The effectiveness of the proposed R-HEFS method is evaluated and analyzed by using four benchmark classifiers viz., Naïve Bayes (NB), random forest (RF), support vector machine (SVM), and AdaBoost. The proposed R-HEFS method turns out to be effective by removing the non-relevant and redundant features during the process of aggregation of base feature selectors and it assists to increase the classification accuracy. Out of 10 different medical datasets, on 7 datasets, R-HEFS has achieved better average classification accuracy. So, the overall results strongly suggest that the proposed R-HEFS method can reduce the dimension of large medical datasets and may help the physicians or medical experts to diagnose (classify) different diseases with lesser computational complexities.

EARLY DETECTION OF ALZHEIMER’S DISEASE USING DATA MINING: COMPARISON OF ENSEMBLE FEATURE SELECTION APPROACHES

Early Alzheimer's disease detection has become an important research area for many years. Various studies in the field of Alzheimer's disease detection have focused on applying individual feature selection methods. In addition to individual feature selection methods, the ensemble feature selection approach has become a creative field. It advocates the combination of the ranked features from various feature selection methods to obtain better results than the current approaches. Thus, this study aims to build a predictive model for early diagnosis of Alzheimer's disease using the ensemble feature selection approaches. Also, Alzheimer's disease dataset consists of three target classes: Normal (CN), Mild Cognitive Impairment (MCI) and Alzheimer's disease (AD). In this study, homogeneous and heterogeneous ensemble approaches have been applied in the feature selection process. Two feature subsets are created based on these ensemble feature selection approaches. A predictive model for early diagnosis of Alzheimer's disease has been build applying Random Forest, Artificial Neural Network, Logistic Regression, Support Vector Machine, and Naïve Bayes data mining algorithms. The predictive model uses the two feature subsets applying these algorithms separately. Then, the performance results are compared to determine which ensemble feature selection approach performs better than the other. This study revealed that better performance result is provided applying Random Forest algorithm with feature subset obtained using the heterogeneous ensemble feature selection approach (91%).

Ensemble feature selection approach for imbalanced textual data using MapReduce

Ensemble feature selection approach for imbalanced textual data using MapReduce

Ensemble Feature Selection for Breast Cancer Classification using Microarray Data

For breast cancer data classification, we propose an ensemble filter feature selection approach named ‘EnSNR’. Entropy and SNR evaluation functions are used to find the features (genes) for the EnSNR subset. A Genetic Algorithm (GA) generates the classification ‘model’. The efficiency of the ‘model’ is validated using 10-Fold Cross-Validation re-sampling. The Microarray dataset used in our experiments contains 50,739 genes for each of 32 patients. When our proposed ‘EnSNR’ subset of features is used; as well as giving an enhanced degree of prediction accuracy and reducing the number of irrelevant features (genes), there is also a small saving of computer processing time.

Ensemble feature selection approach based on feature ranking for rice seed images classification

In smart agriculture, rice variety inspection systems based on computer vision need to be used for recognizing rice seeds instead of using technical experts. In this paper, we have investigated three types of local descriptors, such as Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG) and GIST to characterize rice seed images. However, this approach raises the curse of dimensionality phenomenon and needs to select the relevant features for a compact and better representation model. A new ensemble feature selection is proposed to represent all useful information collected from different single feature selection methods. The experimental results have shown the efficiency of our proposed method in terms of accuracy.